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Abstract. Integer factorization is a very hard computational problem. Cur- 
rently no efficient algorithm for integer factorization is publicly known. How- 
ever, this is an important problem on which it relies the security of many real 
world cryptographic systems. 

I present an implementation of a fast factorization algorithm on MapRe- 
duce. MapReduce is a programming model for high performance applications 
developed originally at Google. The quadratic sieve algorithm is split into the 
different MapReduce phases and compared against a standard implementation. 



1. Introduction 

The security of many cryptographic algorithms relies on the fact that factoring 
large integers is a very computationally intensive task. In particular RSA [1_ would 
be vulnerable if there was an efficient algorithm to factor semiprimes (products of 
two primes). This could have severe consequences, as RSA is one of the most widely 
used algorithms in electronic commerce applications 

There are many algorithms for integer factorization From the trivial trial 
division to the classical Fermat's factorization method [IJ and Euler's factoring 
method ^ to the modern algorithms, the quadratic sieve [6J and the number field 
sieve [7]- In particular the number field sieve algorithm was used in 1996 to factor 
a 512 bit integer 0, the lowest integer length used in commercial RSA implemen- 
tations. There have been several other big integers factored over the course of the 
last decade. I would like to point out that in those cases the feat was accomplished 
with tremendous effort developing the software and a very considerable investment 
in hardware 19J,L10J. 

In what follows I will expose how MapReduce, a distributed computational 
framework, can be used for integer factorization. As an example I will show an 
implementation of the quadratic sieve algorithm. I will also compare in terms of 
performance and cost a conventional implementation with the MapReduce imple- 
mentation. 



2. MapReduce 

I claim no participation in the development of the MapReduce framework. This 
section is basically a short extract of the original MapReduce paper by Jeff Dean 
and Sanjay Ghemawat |11| . MapReduce is a programming model inspired in com- 
putational programming. Users can specify two functions, map and reduce. The 
map function processes a series of (key, value) pairs, and outputs intermediate (key, 
value) pairs. The system automatically orders and groups all (key, value) pairs for 
a particular key, and passes them to the reduce function. The reduce function re- 
ceives a series of values for a single key, and produces its output, which is sometimes 
a synthesis or aggregation of the intermediate values. 

1 
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The canonical example of a MapReduce computation is the construction of an 
inverted index. Let's take a collection of documents T) = {Dq, -Di, -Dat} which 
are composed of words = (c?o.Oi c'q.Ij c^o.Lq) : = (c^i.O; f^i,!: f^i.Li) a-nd so 
on. We define a map function the following way: 

map : (i,Di) -> {{d^,o, (z,0)) , (i, 1)) , {d,^L,, {hLi))} 

that is, for a given document it processes each word in the document and outputs 
an intermediate pair. The key is the word itself, and the value is the location in 
the corpus, indicated as (document, position). The reduce function is defined as: 

reduce : {{d, (ii, ji)) , [d, {iL,jL))} ^ ji) , («L,jL)}) 

For a collection of pairs with the same key (the same word), it outputs a new pair, 
in which the key is the same, and the value is the aggregation of the intermediate 
values. In this case, the set of locations (document and position in the document) 
in which the word can be found in the corpus. 

The MapReduce implementation automatically takes care of the parallel exe- 
cution in a distributed system, data transmission, fault tolerance, load balancing 
and many other aspects of a high performance parallel computation. The MapRe- 
duce model escales seamlessly to thousands of machines. It is used continously for 
a multitude of real world applications, from machine learning to graph computa- 
tions. And most importantly the effort required to develop a high performance 
parallel application with MapReduce is much lower than using other models, like 
for example MPI [H]. 

3. Quadratic Sieve 

The Quadratic Sieve algorithm was conceived by Carl Pomerance in 1981. A 
detailed explanation of the algorithm can be found in jl3j . Here we will just re- 
view the basic steps. Let N be the integer that we are trying to factor. We 
win attempt to find a,b such that: N \ (a'^ - b^) ^ N \ {a + b){a~b). If 
{(a -I- b, N) ,{a — b, N)} ^ {1, N} then we will have a factorization of N. 

Lets define: 

Q{x) = x^ -N 

if we find Xi, X2, ...Xk such that YliLi Q i^i) ^ perfect square, then: 
N\f[Q{x,)-(f[x}j ^f[{x^~N)-xlxl...xl 

i=l \i=l I 2=1 

3.1. Finding Squares. Let's take a set of integers ii, ...iXl which are _B-smooth 
(all Xi factor completely into primes < E). One way to look for ii,Z2,...,H/ such 
that rij=i a square is as follows. Let's denote pi the i-th prime number, 

rijli = PjiPh '"PIl ^ square if and only if 2 | a*; for all k ^ = Omod (2). 
For each Xi we will obtain a vector v"^ — v {xi) where — max {k : pj \ Xi} mod (2). 
That is, each component j of is the exponent of pj in the factorization of Xi mod- 
ulo 2. For example, for B — A: 
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It is immediate that: 



= &,v^ = (1,1,0,0) 
2:2 =45,1-2 = (0,0,1,0) 
X3 -75,^3 - (0,1,0,0) 



Then 



M 




Xi. is a square v 



In conclussion, in order to find a subset of xi, ...,0;^ which is a perfect square, 
we just need to solve the linear system: 



( e,\ 

£2 



~^ mod (2) 



3.2. Sieving for smooth numbers. Back to the original problem, we just need 
to find a convenient set {xi, 2:2, Xi} such that {Q (xi) , Q (2:2) , Q (cci)} are 
_B-smooth numbers for a particular B. First of all, lets notice that we don't need to 
consider every prime number < B. If a prime p verifies: p \ Q{x) for some x then: 



p I Q{x) ^p\x'^-N^x^ = N mod {p) ^ ^— ^ 



1 



Because is a quadratic residue modulo p if and only if the Legendre symbol 
of n over p is 1. We will take a set of primes which verifies that property and we 
will call it factor base. 

In order to consider smaller values of Q(x) we will take values of x around -v/iV, 

i.e. X G 



[\/N\ - M, IVN\ + M for some M. Both B above and M here are 

chosen as indicated in |13| . 

In order to factor all the Q{xi) we will use a method called sieving which is 
what gives the quadratic sieve its name. Notice that p \ Q{x) p \ Q{x + kp) = 

+ 2kpx + Pp'^ - N ^ {x'^ - N) + p (2kx + k'^p) . Then 

Q{x) =0 mod {p) Vfc e N, Q{x + kp) = mod (p) 

N = mod (p) efficiently 



We can solve the equation Q{x) = Qmod{p) <^=> x^ 
and obtain two solutions si, S2 1141 . If we take: 



^P,{1.2} = mm <x^ 



|j; e [Vn\ - M, IVn\ + M 



^{1,2} 



mod 



ip)} 



then all Q (zp^{i_2} + kp) ,kG [0, K] are divisible by p. We can divide each one of 
them by the highest power of p possible. For example: 
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{x^)= (...,6,7,8,9,10,...) 
(Q(x,))= (...,-41,-28,-13,4,23,...) 

(^) = 1 as 77 = 1 = 1^ mod (2) 
a;2 - 77 = mod (2) yields 1, 3, 5, 7, 9, ... 

(. . . , -41, -7, -13, 1, 23, . . .) after sieving by 2 

After sieving for every appropriate p, all the Q{z) that are equal to 1 are smooth 
over the factor base. 



4. Method 

I developed a basic implementation of the Quadratic Sieve MapReduce which 
runs on Hadoop [T3] . Hadoop is an open source implementation of the MapReduce 
framework. It is made in Java and it has been used effectively in configurations 
ranging from one to a few thousand computers. It is also available as a commercial 
cloud service |16| . 

This implementation is simply a proof of concept. It relies too heavily on the 
MapReduce framework and it is severy bound by 10. However the size and complex- 
ity of the implementation are several orders of manitude lower than many competing 
alternatives. 

The 3 parts of the program are : 

• Controller: Is the master job executed by the platform. It runs before 
spawning any worker job. It has two basic functions: first it generates the 
factor base. The factor base is serialized and passed to the workers as a 
counter. Second it generates the full interval to sieve. All the data is stored 
in a single file in the distributed Hadoop file system [IT]. It then relies on 
the MapReduce framework to automatically split it in an adequate number 
of shards and distribute it to the workers 

• Mapper: The mappers perform the sieve. Each one of them receives an 
interval to sieve, and they return a subset of the elements in that input 
sieve which are smooth over the factor base. All output elements of all 
mappers share the same key 

• Reducer: The reducer receives the set of smooth numbers and attempts 
to find a subset of them whose product is a square by solving the system 
modulo 2 using direct bit manipulation. If it finds a suitable subset, it tries 
to factor the original number, N. In general there will be many subsets 
to choose from. In case that the factorization is not succesful with one of 
them, it proceeds to use another one. The single output is the factorization 

In order to compare performance I developed another implementations of the Qua- 
dratic Sieve algorithm in Maple. Both implementations are basic in the sense that 
they implement the basic algorithm described above and the code has not been heav- 
ily optimized for performance. There are many differences between the two frame- 
works used that could impact performance. Because of that a direct comparison of 
running times or memory space may not be meaningful. However it is interesting to 
notice how each of the implementations scales depending on the size of the problem. 



The source code is available online at http: / /www.javiertordable.com/ research 
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Decimal 


Sieve 


MapReduce 


Maple 


Digits 


Size 


Time (s) 


Memory (MB) 


Time (s) 


Memory (MB) 


10 


5832 


2.0 


149.6 


0.1 


7.5 


15 


85184 


3.0 


397.1 


3.5 


15.5 


20 


970299 


35.0 


463.1 


116.0 


100.8 


25 


7529536 


495.0 


670.0 


3413.7 


894.0 



Table 1 . Absolute performance of the MapReduce and Maple implementations 



Decimal 


Sieve 


MapReduce 


Maple 


Digits 


Size 


Time 


Memory 


Time 


Memory 


10 


1.0 


1.0 


1.0 


1.0 


1.0 


15 


14.6 


1.5 


2.7 


35.0 


2.1 


20 


166.4 


17.5 


3.1 


1160.0 


13.4 


25 


1291.1 


247.5 


4.5 


34137.0 


119.2 



Table 2. Normalized performance of the MapReduce and Maple implementations 



Decimal 


Absolute Sieve 


Relative Sieve 


Absolute 


Relative 


Digits 


Size 


Size 


Disk (MB) 


Disk (MB) 


10 


5832 


1.0 


0.1 


1.0 


15 


85184 


14.6 


2.1 


14.6 


20 


970299 


166.4 


29.4 


166.4 


25 


7529536 


1291.1 


275.3 


1291.1 



Table 3. Disk usage of the MapReduce implementation 



5. Results 

Figures 1 and 2 show the results both in absolute terms and normalized. Figure 
3 shows the disk usage of the MapReduce implementation. To test both implemen- 
tations I took a set of numbers of different sizesfl- The number of decimal digits d 
is indicated in the first column of each table. In order to contruct those numbers I 
took two factors close to 10^, with their product slightly over lO'*. 

In each table sieve size indicates the number of elements that the algorithm 
analyzed in the sieve phase. For the MapReduce application the time result is 
taken from the logs, and the memory result is obtained as the maximum memory 
used by the process. For the Maple implementation both time and memory data 
are taken from the on screen information in the Maple environment. Finally disk 
usage data for the MapReduce is taken as the size of the file that contains the list of 
numbers to sieve. The Maple program runs completely in memory for the samples 
analyzed. 



1164656837, 117375210056563, 10446257742110057983, 1100472550655106750000029 
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6. Discussion 

The MapReduce implementation has a relatively big setup cost in time and 
memory when compared with an application in a conventional mathematical envi- 
ronment. However it scales better with respect to the size of the input data. 

MapReduce is optimized to split and distribute data form disk. If an application 
handles a significant volume of data, 10 capacity and performance can be a limiting 
factor. In our case disk usage is directly proportional to the size of the sieve set, 
which grows exponentially on the number of digits. 

Both MapReduce and Maple implementations are similar in terms of develop- 
ment effort. The Maple implementation seems more adequate for small-sized prob- 
lems while the MapReduce application is more efficient for medium-sized problems. 
Also it will be easier to scale in order to solve harder problems. 
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